fix(release): an expired fleet token must not take the whole fleet offline - #790
Merged
Merged
Conversation
…fline FLEET_DISPATCH_TOKEN was set on 2026-08-21 and expired thirty days later. From 2026-09-20 every release run in the fleet failed on "Open the version-bump pull request" with HTTP 401: Bad credentials, and because that step sits above packaging, tagging and the GitHub release, nothing shipped anywhere for two days. Confirmed on portaliq, dossiq, decidiq, openregister and thematiq, all with the same 401. Two defects, not one. The token fallback only covered an ABSENT secret. `secrets.X || github.token` tests for the empty string, and a secret that still exists and no longer authenticates passes that test, so GH_TOKEN carried a dead credential. The step now probes the token against the repository before trusting it and degrades to GITHUB_TOKEN on failure, exactly as it already did when the secret was missing. The follow-on warning now says which of the two happened; "is not set here" would have sent a reader looking for a missing secret that was present and expired. And the step aborted the release rather than the tidy-up. Its own comments claimed twice that it "never fails the release" because "the tag and the package are already published by the time this runs" -- which is simply not where this step sits. That belief is why nobody handled the failure. `gh pr create` now reports an error annotation and returns 0, and the two comments say what is actually true. Verified by replaying the step under a stubbed gh: with the pre-change step and a 401 the step exits 1, with this change it exits 0 and prints the error. Controls: a valid token keeps the PAT and prints nothing, an absent secret still prints the original warning.
The sync step carries the same `secrets.X || github.token` fallback and the same blind spot. It cannot fail the job -- every command there handles its own failure -- but with a dead PAT it opens no sync pull request at all, so `development` silently stops catching up with `beta` and the next promotion conflicts on the version file, which is the drift this step exists to prevent.
Contributor
Author
|
Added the same probe to Sync the release branch back into development, which carries the identical |
rubenvdlinde
deleted the
fix/release-survives-an-expired-fleet-token
branch
September 22, 2026 13:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What broke
FLEET_DISPATCH_TOKEN(organisation secret, set 2026-08-21) expired thirty days later. Since 2026-09-20 every release run in the fleet has failed on Open the version-bump pull request:Confirmed on portaliq, dossiq, decidiq, openregister and thematiq. That step runs above packaging, tagging and the GitHub release, so nothing has shipped anywhere for two days. The job list read
Open the version-bump pull request: failureand every step below it-.Two defects
The fallback only covered an absent secret.
secrets.FLEET_DISPATCH_TOKEN || github.tokentests for the empty string. A secret that still exists and no longer authenticates passes that test, soGH_TOKENcarried a dead credential. The step now probes the token against the repository first and degrades toGITHUB_TOKENon failure, the same way it already handled a missing secret. The follow-on warning says which of the two happened: "is not set here" would have sent a reader looking for a secret that was present and expired.The step aborted the release, not the tidy-up. Its own comments claimed twice that it never fails the release because "the tag and the package are already published by the time this runs". That is not where this step sits, and believing it is why the failure was never handled.
gh pr createnow emits an error annotation and returns 0; the two comments now describe the real order.Verification
The step was replayed with a stubbed
gh:GITHUB_TOKENyaml.safe_loadparses the file andbash -naccepts the step body with the GitHub expressions stubbed.Still needs a human
This makes a dead token survivable, not correct.
FLEET_DISPATCH_TOKENstill has to be rotated at the organisation level. Until it is, bump pull requests open withGITHUB_TOKENand their checks do not run, which is the degraded path this workflow documents at length.